Character recognition processing device, character recognition processing method, and mobile terminal device

ABSTRACT

It is an object of the present invention to improve a rate of recognition and recognition speed, reduce a degree of a specification required for a device forming a photographing device or a system and lower consumed electric power and a cost. In a character recognition processing method according to the present invention, a character image as an object whose character is to be recognized is displayed on a display ( 12 ) together with a cursor of a character frame for recognizing a character. While the cursor is set to a character string as an object to be recognized, the character image is photographed by a photographing part ( 9 ). In an image fetching part ( 3 ), the image data of the photographed character image is fetched. In a layout analyzing part ( 5 ), the fetched image data of the character image and cursor position information from a cursor control part ( 4 ) corresponding to the character image are received. The image data is collated with the cursor position information to analyze the arrangement of the lines or characters of the character string as the object to be recognized. Then, in a character cutting part ( 6 ), the character is cut on the basis of the analyzed result. The character is recognized relative to the character image by a character recognizing part ( 7 ) to convert the image data to character data.

TECHNICAL FIELD

The present invention relates to a character recognition processingdevice, a character recognition processing method and a portableterminal device that are preferably used for a portable terminal devicesuch as a portable telephone, a portable information terminal or thelike.

BACKGROUND ART

As a usual character recognizing method and a usual characterrecognizing device, a character recognizing method and a characterrecognizing device that are disclosed in, for instance, JP-A-2001-60250(Patent document 1) and JP-A-2000-76378 (Patent document 2) have beenknown.

In the character recognizing method and the character recognizing devicedisclosed in JP-A-2001-60250, a feature amount is extracted from acharacter whose image is inputted and the character is recognized on thebasis of the extracted feature amount. In this example, every time thecharacter whose image is inputted is scanned at prescribed intervals,the number of times of the intersections of scanning lines and thecharacter are obtained. Further, the sum total of the differences of theobtained number of times of the intersections between the respectiveadjacent scanning lines is obtained as one of the feature amount torecognize the character. Scanning operations relative to the characterto be recognized are carried out at prescribed intervals in at least twodifferent directions or carried out at prescribed intervals in twodirections intersecting at right angles with each other.

On the other hand, in the character recognizing method disclosed inJP-A-2000-76378, the outline data of inputted image data is formed. Dataobtained by normalizing the formed outline data is collated withdictionary data formed by extracting the outline data of a previouslyregistered standard character to recognize a character. At this time,depending on the typeface or the distortion of the inputted image data,the outline data of the image data is deformed to a typeface that can berecognized by the dictionary data or deformed so that the distortion iscorrected before the data is collated with the dictionary data.

[Patent Document]

-   JP-A-2001-60250    [Patent Document 2]-   JP-A-2000-76378

However, in the usual character recognizing method and the characterrecognizing device, when the character is recognized from fetchedcharacter image data, a special process has needed to be carried out byusing scanning lines or outline data respectively in the analysis of alayout or the cutting operations of lines and the character to improve arate of recognition or recognition speed.

Further, ordinarily, when a character string whose characters are to berecognized is entirely photographed at a time, the number of pixels of aphotographing device (for instance, a CCD camera or the like) needs tobe increased to ensure a resolution for one character and large-scaledata of a necessary value or more is treated. Thus, a host CPU or a MIPS(Million Instructions Per Second) value of a DSP or a work RAMundesirably needs to be increased.

The present invention is devised by considering the above-describedcircumstances and an object of the present invention is to provide acharacter recognition processing device, a character recognitionprocessing method and a portable terminal device that can improve a rateof recognition and a recognition speed, can reduce the degree of aspecification required for a device forming a photographing device or asystem and can decrease consumed electric power and a cost.

DISCLOSURE OF THE INVENTION

A character recognition processing device according to the presentinvention comprises an image fetching unit for fetching the image dataof a character image as an object to be recognized that is photographedfor recognizing a character; a cursor information output unit foroutputting cursor position information showing the position of acharacter frame for recognizing the character coordinated with thecharacter image; a layout analyzing unit for collating the cursorposition information with the fetched image data of the character imageto analyze the arrangement of the character; a character cutting unitfor extracting the character image on the basis of the analyzed resultof the layout analyzing unit; and a character recognizing unit forrecognizing the extracted character image as the character andconverting the character image to character information.

According to the above-described structure, when the character of theobject whose character is to be recognized is recognized, the characterimage as the object to be recognized is displayed on a display partformed with a liquid crystal display element together with the characterframe for recognizing the character. While a cursor of the characterframe for recognizing the character is set to a character string as anobject to be recognized, the character image is photographed by an imagepick-up part such as a CCD camera. Then, the image data of the characterimage is fetched by the image fetching unit. The cursor positioninformation is collated with the fetched image data of the characterimage by the layout analyzing unit to analyze the arrangement of linesor characters. The character is extracted and recognized relative to thecharacter image by the character cuting unit and the characterrecognizing unit. Accordingly, in the layout analyzing unit, thearrangement information of the lines or the characters of the characterstring as the object to be recognized can be precisely analyzed by usingthe cursor position information. As a result, a rate of recognition anda recognition speed can be improved. Further, even when the number ofpixels of the image data for recognizing the character is not increased,the character can be highly accurately recognized by using the cursorposition information. Accordingly, a specification of high performancethat a photographing part has many pixels is not required and aspecification required for the device can be reduced. Further, thethroughput of the data in the device can be reduced, so that aspecification necessary for a device forming a system such as a CPU, aDSP, a memory or the like can be reduced. Thus, the degree of thespecification of each of parts of the device can be reduced, so that aconsumed electric power and a cost can be lowered.

Further, in the above-described structure, when the character image asthe object to be recognized is composed of a plurality of characterimages obtained by a continuous photographing operation, the imagefetching unit fetches the image data of the character imagesrespectively for prescribed areas from all of the character images. Thelayout analyzing unit collates the cursor position information with eachof the fetched image data of the plurality of character images toanalyze the arrangement of the characters.

In the above-described structure, when the entire part of the characterimage cannot be photographed at a time by a photographing operation, forinstance, when the character string as the object whose characters areto be recognized is large, while the photographing part is moved, acontinuous photographing operation is carried out to photograph thecharacter image. In the obtained image data of the character image, theimage data of the character image is fetched respectively for prescribedareas from the entire part of the character image by the image fetchingunit. The cursor position information is collated with each of thefetched image data of the plurality of character images by the layoutanalyzing unit to analyze the arrangement of the lines or thecharacters. As described above, the character image is divided intostill images of some frames and the still images are fetched togetherwith the cursor position information. Consequently, even when thecharacter string as the object to be recognized is long, the arrangementinformation of the character string as the object to be recognized canbe precisely analyzed by employing the cursor position information inthe layout analyzing unit.

Further, in the above-described structure, when the character image asthe object to be recognized is composed of a plurality of characterimages obtained by a continuous photographing operation, the imagefetching unit fetches the image data of the character imagesrespectively for prescribed areas from all of the character images. Thelayout analyzing unit collates the cursor position information with theimage data in which the plurality of fetched character images areconnected together to analyze the arrangement of the characters.

In the above-described structure, when the entire part of the characterimage cannot be photographed at a time by a photographing operation, forinstance, when the character string as the object whose characters areto be recognized is large, while the photographing part is moved, acontinuous photographing operation is carried out to photograph thecharacter image. In the obtained image data of the character image, theimage data of the character image is fetched respectively for prescribedareas from the entire part of the character image by the image fetchingunit. After the fetched image data of the plurality of character imagesis temporarily connected to one image, the image data is collated withthe cursor image information to analyze the arrangement of lines orcharacters by the layout analyzing unit. As described above, thecharacter image is divided into still images of some frames and thestill images are fetched together with the cursor position information.Consequently, even when the character string as the object to berecognized is long, the arrangement information of the character stringas the object to be recognized can be precisely analyzed by employingthe cursor position information in the layout analyzing unit.

Further, the present invention provides a portable terminal devicehaving the character recognition processing device as described above.

According to the above-described structure, in the portable terminaldevice on which the character recognition processing device is mounted,while a rate of recognizing and a recognition speed for recognizing acharacter is improved, a specification required for a device forming asystem such as a photographing part, a CPU, a DSP, a memory or the likecan be reduced. Thus, the degree of the specification of each of partsof the device can be reduced and a low consumed electric power and a lowcost can be realized.

A portable terminal device according to the present invention comprisesa photographing unit for photographing a character image as an object tobe recognized for recognizing a character; an image fetching unit forfetching the image data of the photographed character image; a cursorinformation output unit for outputting cursor position informationshowing the position of a character frame for recognizing the charactercoordinated with the character image; a layout analyzing unit forcollating the cursor position information with the fetched image data ofthe character image to analyze the arrangement of the character; acharacter cutting unit for extracting the character image on the basisof the analyzed result of the layout analyzing unit; and a characterrecognizing unit for recognizing the extracted character image as thecharacter and converting the character image to character information.

According to the above-described structure, the layout of the charactercorresponding to the character frame for recognizing the character isanalyzed on the basis of the image data of the character image and thecursor position information showing the position of the character framefor recognizing the character to recognize the character. Thus, while arate of recognizing and a recognition speed for recognizing thecharacter is improved, a specification required for a device forming asystem such as a photographing part, a CPU, a DSP, a memory or the likecan be reduced. Accordingly, the degree of the specification of each ofparts of the device can be reduced and a low consumed electric power anda low cost can be realized.

Further, in the above-described structure, a character frame displayunit is further provided for displaying the character frame forrecognizing the character overlapped on the character image uponphotographing by the photographing unit.

According to this structure, while a user sees the displayed characterframe for recognizing the character, the user can easily photograph thecharacter as the object to be recognized to the character frame forrecognizing the character to recognize the character.

In any of the above-described structures, a recognized character displayunit is further provided for displaying the character information as arecognized result by the character recognizing unit.

According to this structure, the user can easily recognize the characterinformation of the recognized result by a display.

Further, in the above-described structure, the recognized characterdisplay unit individually selectively displays the character informationas the recognized result for each of prescribed character units.

According to this structure, the user can individually select andutilize the recognized character information for each unit such as thetypes of the characters on the basis of the displayed characterinformation.

Further, in any of the above-described structures, a recognizedcharacter storing unit is further provided for storing the characterinformation as the recognized result by the character recognizing unit.

According to this structure, the user can store and utilize therecognized character information in a desired form.

Further, in the above-described structure, the recognized characterstoring unit stores the character information in a recognized characterstoring area.

According to this structure, for instance, the character information ofa plurality of recognized results can be stored in the recognizedcharacter storing area and used.

Further, in the above-described structure, the recognized characterstoring unit registers the character information in a data basecorresponding to a type of each character information when the type ofthe character information is any of a telephone number, a mail address,a URL (Uniform Resource Location).

According to this structure, when the character of the telephone number,the mail address, the URL or the like is recognized, the characterinformation can be registered in the data base such as a telephonedirectory or a book mark depending on the type of the characterinformation and utilized.

In any of the above-described structures, a recognized characterutilizing unit is further provided for utilizing the characterinformation as the recognized result by the character recognizing unitin accordance with the type of the character information.

According to this structure, the user can effectively employ thecharacter information for each of the types such as the telephonenumber, the mail address, the URL or the like in accordance with thetype of the recognized character information.

Further, in the above-described structure, when the type of thecharacter information is the telephone number, the recognized characterutilizing unit displays a transmitting screen to the telephone number.

According to this structure, when the recognized character informationis the telephone number, the transmitting screen to the telephone numberis displayed so that the user can simply carry out a transmittingoperation.

In the above-described structure, when the type of the characterinformation is the mail address, the recognized character utilizing unitdisplays an electronic mail preparing screen to the mail address.

According to this structure, when the recognized character informationis the mail address, the electronic mail preparing screen to the mailaddress is displayed so that the user can simply prepare and transmit anelectronic mail.

Further, in the above-described structure, when the type of thecharacter information is the URL (Uniform Resource Locator), therecognized character utilizing unit displays a network connecting screento the URL.

According to this structure, when the recognized character informationis the URL, the network connecting screen to the URL is displayed sothat the user can simply carry out a connecting operation to a networksuch as the internet to obtain desired information.

Further, in any of the above-described structures, the photographingunit has a function for continuously photographing the character imageas the object to be recognized. When a plurality of the character imagesas the objects to be recognized is obtained by the photographing unit,the image fetching unit fetches the image data of the plurality ofcharacter images that are connected together. The character recognizingunit recognizes the characters of the image data of the connectedcharacter images to convert the image data to the character information.

According to this structure, when the character string as the object tobe recognized is not put in a photographing range at a time, thecharacter images are continuously photographed by coordinating thecharacter images with the character frame for recognizing the characterand the plurality of the character images thus obtained is connectedtogether. Thus, even when the character string is long, a rate ofrecognition and a recognition speed for recognizing the character can beimproved.

In any of the above-described structures, the photographing unit has afunction for continuously photographing the character image as theobject to be recognized. When a plurality of the character images as theobjects to be recognized is obtained by the photographing unit, theimage fetching unit fetches the image data of the plurality of characterimages. The character recognizing unit recognizes the characters of thefetched image data respectively for the plurality of character images toconvert the image data to the character information.

According to this structure, when the character string as the object tobe recognized is not put in a photographing range at a time, thecharacter images are continuously photographed by coordinating thecharacter images with the character frame for recognizing the character.The characters of the image data are recognized respectively for theplurality of character images thus obtained. Thus, even when thecharacter string is long, a rate of recognition and a recognition speedfor recognizing the character can be improved.

Further, in any of the above-described structures, the characterrecognizing unit has a plurality of recognizing modes that meet thetypes of the characters as the objects to be recognized and carries outa character recognizing process suitable for the corresponding type ofthe character in accordance with a preset recognizing mode.

According to this structure, the user sets the recognizing modecorresponding to the type of the character that is to be recognized bythe user. Thus, the character recognizing process suitable for thecorresponding type of the character can be carried out to improve therate of recognition.

A character recognition processing method according to the presentinvention comprises: a step of displaying and photographing a characterimage as an object to be recognized together with a character frame forrecognizing a character; a step of fetching the image data of thephotographed character image; a step of outputting cursor positioninformation showing the position of the character frame for recognizingthe character coordinated with the character image; a step of collatingthe cursor position information with the fetched image data of thecharacter image to analyze the arrangement of the character; a step ofextracting the character image on the basis of the analyzed result ofthe arrangement of the character; and a step of recognizing theextracted character image as the character and converting the characterimage to character information.

According to this method, the arrangement information of lines orcharacters of a character string as an object to be recognized can beprecisely analyzed by using the cursor position information to improve arate of recognition or a recognition speed. Further, the cursor positioninformation is used so that a specification required for each part ofthe device can be reduced. Since the throughput of the data in thedevice can be reduced, the degree of a specification of the device canbe reduced and a low consumed electric power and a low cost can berealized.

Further, the present invention provides a character recognitionprocessing program in which the above-described steps are respectivelyexecuted by a computer.

According to this program, the layout of the character corresponding tothe character frame for recognizing the character can be preciselyanalyzed on the basis of the image data of the character image and thecursor position information showing the position of the character framefor recognizing the character to recognize the character. While the rateof recognition and the recognition speed for recognizing the characterare improved, the degree of the specification of each part of the devicecan be reduced and a low consumed electric power and a low cost can berealized.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing the structure of a portable terminaldevice having a character recognition processing device according to afirst embodiment of the present invention;

FIG. 2 is an explanatory view of an operation during a characterrecognizing process in the character recognition processing deviceaccording to the first embodiment of the present invention showing astate that a character string as an object to be recognized isphotographed;

FIG. 3 is an explanatory view of an operation during a characterrecognizing process in the character recognition processing deviceaccording to a second embodiment of the present invention showing astate that a character string as an object to be recognized isphotographed;

FIG. 4 is an explanatory view of an operation during a characterrecognizing process in the character recognition processing deviceaccording to the second embodiment of the present invention showing astate of a character recognizing process relative to a character stringas an object to be recognized that is divided into a plurality of parts;

FIG. 5 shows views showing the structure of an external appearance of aportable telephone according to a third embodiment of the presentinvention, FIG. 5(A) being a front view and FIG. 5(B) being a rear view;

FIG. 6 is a block diagram showing the structure of the portabletelephone according to the third embodiment;

FIG. 7 is a block diagram showing a functional structure related to acharacter recognizing process in the portable telephone according to thethird embodiment;

FIG. 8 is an explanatory view showing a functional structure of acharacter using part in the structure shown in FIG. 7;

FIG. 9 is an explanatory view showing a flow of all of processes relatedto a character recognition and the use of characters in the thirdembodiment;

FIG. 10 is a diagram showing one example of an object whose character isto be recognized;

FIG. 11 is a diagram showing one example of a main menu screen of an OCRfunction in the third embodiment;

FIG. 12 shows one example of each screen of the OCR function in thethird embodiment, FIG. 12(A) showing a preview display screen, FIG.12(B) showing a camera function setting screen and FIG. 12(C) showing arecognizing mode setting screen, respectively; and

FIG. 13 shows one example of each screen of the OCR function in thethird embodiment, FIG. 13(A) showing a recognized character displayscreen, FIG. 13(B) showing a function menu display screen, FIG. 13(C)showing an audio transmitting screen, FIG. 13(D) showing an internetconnecting screen and Fig, and 13(E) showing a mail preparing screen,respectively.

In the drawings, reference numeral 1 designates a portable terminaldevice. 2 designates a character recognition processing device. 3designates an image fetching part. 4 designates a cursor control part. 5designates a layout analyzing part. 6 designates a character cuttingpart. 7 designates a character recognizing part. 8 designates a formatoutput part. 9 designates a photographing part. 10 designates a displaycontrol part. 11 designates an application part. 12 designates adisplay. 30 designates a portable telephone. 33 designates a firstdisplay part. 34 designates a camera. 35 designates a second displaypart. 36 designates an operating key. 41 designates a photographingpart. 42 designates an operating part. 43 designates a display part. 44designates a communication part. 45 designates an audio processing part.46 designates a storing part. 47 designates a control processing part.51 designates a character recognizing part. 52 designates a characterusing part. 61 designates a temporarily storing part. 62 designates astoring part. 63 designates an output part. 64 designates a typedeciding part.

BEST MODE FOR CARRYING OUT THE INVENTION

Now, referring to the drawings, embodiments of the present inventionwill be described below.

First Embodiment

FIG. 1 is a block diagram showing the structure of a portable terminaldevice having a character recognition processing device according to afirst embodiment of the present invention. In this embodiment, astructural example that the character recognition processing device 2 ismounted on the portable terminal device 1 is described.

The character recognition processing device 2 of this embodimentincludes an image fetching part 3, a cursor control part 4, a layoutanalyzing part 5, a character cutting part 6, a character recognizingpart 7 and a format output part 8. These component elements are realizedby the operation of a software program, the combination of software andhardware or an exclusive hardware in a control processing part includinga processor and a memory or the like.

The portable terminal device 1 includes a photographing part 9 composedof, for instance, a CCD camera, a display control part 10 and anapplication part 11 in addition to the character recognition processingdevice 2. The photographing part 9 photographs an object whose characteris to be recognized and fetches the object as photographed data to inputthe photographed data to an image fetching part 3 of the characterrecognition processing device 2. The image fetching part 3 correspondsto an image fetching unit to convert the inputted photographed data toimage data of a prescribed format and input the image data to thedisplay control part 10. The cursor control part 4 corresponds to acursor information output unit to input cursor position informationrelated to a cursor that functions as a character frame for recognizinga character to the display control part 10. The display control part 10converts the inputted image data and the cursor position information todisplay data, further processes the display data to a format suitablefor a display 12 composed of a liquid crystal display element andoutputs the processed display data to display information on the display12.

The layout analyzing part 5 corresponds to a layout analyzing unit toreceive the image data of the prescribed format from the image fetchingpart 3 and the cursor position information from the cursor control part4 upon recognizing the character. Then, the layout analyzing part 5collates the cursor position information with the image data to analyzea layout. The character cutting part 6 corresponds to a charactercutting unit to cut and extract the image data for a character unit onthe basis of the analyzed result of the layout. The characterrecognizing part 7 corresponds to a character recognizing unit torecognize the cut image data for a character unit and convert the imagedata to character data. The format output part 8 converts the characterdata recognized in the character recognizing part 7 to a proper format.The application part 11 uses the character data converted to the formatin the format output part 8 as data for an application and inputs thedata to the display control part 10 as required to display the data onthe display 12.

Now, referring to FIG. 2, an operation of the character recognitionprocessing device of the first embodiment will be described below. FIG.2 is an explanatory view of an operation at the time of a characterrecognizing process according to the first embodiment and shows a statethat a character string as an object to be recognized is photographed.

A character string 17 as an object to be recognized in an object 16whose character is to be recognized is photographed by the photographingpart 9 of the portable terminal device 1 to recognize the character.When the photographed character string 17 as the object to be recognizedis displayed on the display 12, a cursor 18 as a character frame forrecognizing a character is displayed together with the character string17 as the object to be recognized. While a user adjusts a photographingposition, the user sets the character string 17 as the object to berecognized to the cursor 18, the user operates an operating button 19 tophotograph a still image under this state.

The photographed still image of the character string 17 as the object tobe recognized is converted to the image data of a determined format andfetched in the image fetching part 3 and the image data is supplied tothe layout analyzing part 5. The layout analyzing part 5 receives thecursor position information from the cursor control part 4 to collatethe cursor position information with the image data of a characterimage. In such a way, the layout analyzing part 5 can precisely analyzethe arrangement information of lines and characters of the characterstring 17 as the object to be recognized by using the cursor positioninformation. The character cutting part 6 cuts the image data for acharacter unit. The image data extracted for a character unit isconverted to the character data to recognize the character.

As described above, in the character recognition processing device ofthe first embodiment, when the character of the character string 17 asthe object to be recognized of the object 16 whose character is to berecognized is recognized, the image of the character string 17 as theobject to be recognized and the cursor 18 as the character frame forrecognizing the character are displayed together on the display 12.While the user sets the cursor 18 to the character string 17 as theobject to be recognized, the still image is photographed. Thus, theimage data of the photographed character image is fetched together withthe cursor position information. Thus, in the layout analyzing part 5,the arrangement information of the lines or the characters can be easilyand precisely analyzed by using the cursor position information so thata rate of recognition and a recognition speed can be improved inrecognizing the character.

Second Embodiment

Now, an operation of a character recognition processing device accordingto a second embodiment of the present invention will be described byreferring to FIGS. 3 and 4. FIGS. 3 and 4 are explanatory views of anoperation during a character recognizing process according to the secondembodiment. FIG. 3 shows a state that a character string as an object tobe recognized is photographed. FIG. 4 shows a manner of the characterrecognizing process relative to the character string as the object to berecognized that is divided into a plurality of parts. Since thestructure of the character recognition processing device is the same asthat of the first embodiment, FIG. 1 is applied thereto.

In the second embodiment, a character string 23 as an object to berecognized in an object 22 whose character is to be recognized isphotographed by a photographing part 9 of a portable terminal device 1to recognize a character in the same manner as that of the firstembodiment. In this case, when the range of the character string 23 asthe object to be recognized is wide so that an image cannot be fetchedat a time, a below-described operation is carried out.

When the photographed character string 23 as the object to be recognizedis displayed on a display 12, a cursor 18 as a character frame forrecognizing a character is displayed together with the character string23 as the object to be recognized. While a user adjusts a photographingposition, the user sets a part of the position of a start end of thecharacter string 23 as the object to be recognized (a part of a left endin an example shown in FIG. 3) to the cursor 18. Under this state, anoperating button 19 is operated to start a photographing operation of acontinuous still image. After the photographing operation is started,while the user fixes the moving direction of the portable terminaldevice 1, the user slides the portable terminal device to a directionopposite to the position of the start end upon starting a photographingoperation (a right direction in the example shown in FIG. 3) in parallelwith the character string 23 as the object to be recognized so as to bedirected to a writing direction of a character. Thus, when the characterstring 23 as the object to be recognized is displayed on the display 12so as to accommodate the character string 23 in the cursor 18 up to aposition of a finish end (a position of a right end in the example shownin FIG. 3) of the character string 23 as the object to be recognized,the photographing operation of the continuous still image is finished.At this time, the user keeps pressing down the operating button 19, forinstance, during the photographing operation of the continuous stillimage.

The continuous still image of the photographed character string 23 asthe object to be recognized is divided into a plurality of image dataand fetched in an image fetching part 3 as shown in FIG. 4. In thiscase, a character image of one line is divided into still images of someframes (1) to (4) and the still images are extracted from all ofcharacter images and converted to the image data of a determined format.The image data is delivered to a layout analyzing part 5.

Further, in the layout analyzing part 5, cursor position information isreceived from a cursor control part 4 to collate the cursor positioninformation with the image data of the character image. In this case,characters may not be possibly respectively accommodated in thecharacter frame of the cursor with respect to the character images ofeach frame used for analyzing a layout. Accordingly, the cursor positioninformation can be used without a problem to analyze arrangementinformation relative to a “line”. However, the cursor positioninformation may not be used for analyzing the arrangement informationfor each character. In this case, the cursor position information may beshifted to the moving direction of the continuous still image to analyzethe arrangement information of each character. As described above, inthe layout analyzing part 5, even when the character string 23 as theobject to be recognized is long, the arrangement information of thecharacter string 23 as the object to be recognized can be preciselyanalyzed by employing the cursor position information. In thisembodiment, the arrangement information of the lines can be preciselyanalyzed.

Then, in a character cutting part 6, the image data is cut for acharacter unit. In a character recognizing part 7, the extracted imagedata for a character unit is converted to character data to recognize acharacter. In this case, as the recognized result of the character,“ABCDEFGHIJKL” is obtained in the frame (1). In the frame (2),“FGHIJKLMNOP” is obtained. In the frame (3), “JKLMNOPQRSTU” is obtained.In the frame (4), “OPQRSTUVW” is obtained. In the character recognizingpart 7, the recognized results of the characters respectively for theframes are synthesized so that “ABCDEFGHIJKLMNOPQRSTUVW” is obtained asa final recognized result of the characters.

In the second embodiment, for the purpose of simplicity, such anexplanation that the character image for one line is collated with thecursor to perform a series of character recognizing processes isdescribed. However, as shown in FIG. 3, when the cursor 18 is providedfor two lines, a series of character recognizing processes can becarried out for the two lines at the same.

As described above, in the character recognition processing device ofthe second embodiment, when the characters of the character string 23 asthe object to be recognized of the object 22 whose character is to berecognized are recognized, the cursor 18 as the character frame forrecognizing the character is displayed on the display 12 together withthe image of the character string 17 as the object to be recognized.Then, the user moves the portable terminal device 1 to photograph thecontinuous still image. Then, the photographed character image isdivided into some still images and the image data of each frame isfetched together with the cursor position information. Thus, in thelayout analyzing part 5, the arrangement information of the lines can beeasily and accurately analyzed by using the cursor position information,and a rate of recognition and recognition speed can be improved inrecognizing a character.

In the above-described second embodiment, when the continuous stillimage photographing is carried out to obtain a plurality of image data,the portable terminal device is moved on the character string as theobject to be recognized by keeping the operating button 19 of theportable terminal device 1 pressed down. However, the present inventionis not limited thereto. For instance, while the user moves the main bodyof the portable terminal device 1 to move a photographing range, theuser may press down the operating button 19 each time to intermittentlyobtain the plurality of image data. The user may also obtain the imagedata of a continuous wide range as the image of the character string asthe object to be recognized.

Further, the layout analyzing part 5 may analyze the layout for eachcharacter image of a plurality of character images (still images)fetched by the image fetching part 3. The layout analyzing part maytemporarily connect the plurality of character images to one characterimage, and then, analyze the layout.

As described above, in this embodiment, even when the number of pixelsof the image data for recognizing the character is not increased, thecharacter can be highly accurately recognized by using the cursorposition information. Accordingly, a specification of high performance,that the number of pixels of the photographing part 9 is increased, isnot required and a specification required for the device can be reduced.Further, the throughput of the data in the device can be reduced, sothat a specification necessary for the device forming a system such as aCPU, a DSP, a memory or the like can be reduced. Accordingly the degreeof a specification of each part of the device can be reduced and a lowconsumed electric power and a low cost can be realized.

Third Embodiment

Now, one example of a structure and an operation when a characterrecognition processing device is mounted on a portable telephone as athird embodiment will be described below.

FIG. 5 is a view showing the structure of an external appearance of aportable telephone according to the third embodiment of the presentinvention. FIG. 5(A) is a front view and FIG. 5(B) is a rear view. Theportable telephone 30 of this embodiment includes a first casing 31 anda second casing 32. On a front surface side of the first casing 31, thatis, on a surface that mainly faces a user when the user opens the firstcasing 31 and the second casing 32 and holds them, a first display part33 composed of a liquid crystal display element is provided. Further, ina rear surface side opposite to the front surface of the first casing31, a camera 34 as a photographing part and a second display part 35composed of a liquid crystal display element are provided.

On the front surface side of the second casing 32, an operating key 36as an operating part having a plurality of key buttons such as anon-hook/off-hook key, a numeric character key, a function key or thelike is disposed. Various kinds of functions are assigned to thefunction keys PF1 to PF5 depending on operation modes or display screensof the device.

FIG. 6 is a block diagram showing the structure of the portabletelephone according to the third embodiment. The portable telephoneincludes a photographing part 41 having a function of a photographingunit for photographing the image of an object whose character is to berecognized, an operating part 42 for performing an operation and inputby a user, a display part 43 having functions of a character framedisplay unit and a recognized character display unit for displayingvarious kinds of information such as character recognized results, acommunication part 44 for performing a radio communication with a basestation of a mobile communication system, an audio processing part 45for performing an audio process of an audio speech or a call receivingsound, a storing part 46 having a function of a recognized characterstoring unit for storing various kinds of information and a controlprocessing part 47 for carrying out a character recognizing operation,an application process such as the use of recognized character data orcontrolling respective parts.

When the audio communication is performed in the portable telephone, aspeaking voice is inputted, transmitted/received and outputted by thecommunication part 44, the audio processing part 45 and the controlprocessing part 47. When a message such as an electronic mail istransmitted or received or various kinds of information are obtained byaccessing a network such as an internet, character data is inputted andinformation such as the character data or image data is transmitted andreceived and displayed by the operating part 42, the display part 43,the communication part 44 and the control processing part 47. When theimage is photographed, an object to be photographed is photographed toobtain the image data of a photographed image by the photographing part41 and the control processing part 47. The storing part 46 storesvarious kinds of information such as telephone directory registeredinformation including the telephone number or the mail address of acommunication mate, the character data such as an electronic mail, theimage data such as the photographed image, audio data such as a callreceiving sound.

FIG. 7 is a block diagram showing a functional structure related to acharacter recognizing process in the portable telephone according to thethird embodiment. In the third embodiment, a photographing part 41 and acharacter recognizing part 51 similar to those of the first and secondembodiments are provided. Further, a character using part 52 is alsoprovided. The character using part 52 has a function of a recognizedcharacter utilizing unit for using character data as a recognized resultin the function of each part is also provided. The character recognizingpart 51 and the character using part 52 are realized by hardwareincluding a processor and a memory and a software program operating byusing them.

FIG. 8 is an explanatory view showing a functional structure of thecharacter using part in the structure shown in FIG. 7. The characterusing part 52 includes a temporarily storing part 61, a storing part 62and an output part 63. The temporarily storing part 61 is a recognizedcharacter storing unit to temporarily store the character data after thecharacter recognizing process. The storing part 62 is a recognizedcharacter storing unit and has a plurality of storing areas as arecognized character storing area to respectively store a plurality ofcharacter data. The output part 63 has a type deciding part 64 fordeciding the type of the character data to perform an output operationcorresponding to the type of the character data of a recognized result.When the character data obtained by recognizing a character is atelephone number, an operation related to an audio calling to thetelephone number or a registration in a telephone directory as adatabase is carried out. When the character data is a mail address, anoperation related to a preparation of a mail to the mail address or theregistration in the telephone directory is carried out. When thecharacter data is a URL (Uniform Resource Locator), an operation relatedto an internet connection for accessing the URL or a registration in abook mark as a database is carried out.

Now, a specific example of an operation concerning the recognition ofthe character and the use of the character in the third embodiment willbe described below. FIG. 9 is an explanatory view showing the flow of anentire process of the recognition of the character and the use of thecharacter. FIG. 9 shows the transition of each of operating states on adisplay screen displayed on the display part 43 as a center. Here, acase is described in which characters written on a card 101 as theobject whose character is to be recognized as shown in FIG. 10 arephotographed as objects to be recognized and the characters arerecognized by an OCR (Optical Character Recognition) function mounted onthe portable telephone.

When the OCR function of the portable telephone is activated, a mainmenu screen 71 corresponding to an initial screen is displayed. FIG. 11shows one example of the main menu screen 71. On this main menu 71, thelist of the registered contents of the plurality of character data isdisplayed. In this embodiment, a plurality (for instance, 20) ofcharacter data of character strings to be recognized can be registered.Here, when the user selects a column of a “non-registered” to focus it(shown by oblique lines in the drawing) by a highlight display or thelike and presses a “select” button assigned to the function key PF1, thephotographing part 41 is activated to photograph the image of a newcharacter string as an object to be recognized to become a waiting statefor photographing by a camera. Thus, a preview display screen 72 isdisplayed. When a focused position is moved to switch an object to beselected, a cross key assigned to the outer peripheral part of thefunction key PF1 is operated.

FIG. 12(A) shows one example of the preview display screen 72. On thepreview display screen 72, a character image 81 as an object to berecognized that is photographed by the photographing part 41 isdisplayed. On the character image 81, a character frame 82 forrecognizing a character showing a character recognizing area issuperimposed and displayed. When the user views the character image 81and adjusts a photographing range so that the character string as theobject to be recognized is accommodated in the character frame 82 forrecognizing the character and presses down a “photographing” buttonassigned to the function key PF1, the character image 81 isphotographed. Further, a series of character recognizing processesdescribed in the first embodiment such as the analysis of the layout ofthe character string as the object to be recognized, cutting of thecharacter, the recognition of the character or the like are carried outby the character recognizing part 51.

Further, when the user presses down a “function” button assigned to thefunction key PF3 in the preview display screen 72 shown in FIG. 12(A), amode shifts to a function setting mode of the photographing part 41 todisplay a camera function setting screen 73. FIG. 12(B) shows oneexample of the camera function setting screen 73. In the camera functionsetting screen 73, any of a recognizing mode, a brightness adjustmentand a photo-light (photographing light) can be selected as a target tobe set. Here, when the user selects a column of the “recognizing mode”to focus it and presses down the “select” button assigned to thefunction key PF1, a recognizing mode setting screen 74 is displayed.

FIG. 12(C) shows one example of the recognizing mode setting screen 74.In the recognizing mode setting screen 74, when any of an automaticmode, a URL mode, a mail address mode and a telephone number mode isselected as the recognizing mode, a character recognizing processspecified by the selected mode is carried out for all the characterstrings of the photographed character image. When a free mode isselected, the character is recognized relative to an arbitrary characterstring as an object and all the character strings that can be recognizedare outputted as recognized character data. When the automatic mode isselected, the character recognizing part 51 automatically decides asuitable recognizing mode depending on the contents of the characterimage 81 from among the four modes to recognize the character under thesuitable recognizing mode. Here, when the user selects the recognizingmode to focus it and presses down the “select” button assigned to thefunction key PF1, the screen returns to the preview display screen 72.In the above-described operation, the character recognizing processspecified by the selected mode is carried out for all the characterstrings of the photographed character image. However, when any of theautomatic mode, the URL mode, the mail address mode and the telephonenumber mode is selected, only the character string corresponding to theURL or the like may be outputted as the recognized character data.

When the photographing and character recognizing processes are completedfrom the state of the preview display screen 72, a recognized characterdisplay screen 75 is displayed. FIG. 13(A) shows one example of therecognized character display screen 75. In the recognized characterdisplay screen 75, the character data of the character string as arecognized result is displayed. Here, when the recognized characterstring includes the telephone number, the mail address and the URL, ananchor display is carried out in which the above-described items arerespectively underlined to focus a selected character string. In thisembodiment, the recognized character string is analyzed to identify thetelephone number, the mail address and the URL. The telephone number,the mail address and the URL can be individually selected and used inaccordance with the type of the character string.

To identify the telephone number, the mail address and the URL, forinstance, a below-described deciding method is used.

-   (1) Telephone Number

When there is a numeric character having 10 or more figures that startsfrom 0

-   (2) Mail Address

When there is a character including “@”

-   (3) URL

When there is a character starting from “http://” or “https://”

In the main menu screen 71, when the user selects a column of aregistered character string to focus it and presses down the “select”button assigned to the function key PF1, the recognized characterdisplay screen 75 is displayed to display the character data of thecharacter strings recognized and stored in the past.

In the recognized character display screen 75, when the user pressesdown the “function” button assigned to the function key PF3, a functionmenu display screen 76 is displayed. FIG. 13(B) shows one example of thefunction menu display screen 76. In the function menu display screen 76,any of connecting, editing, copying, storing, newly registering in atelephone directory, additionally registering in a telephone directoryand registering a book mark can be selected.

When a long character string that cannot be photographed once iscontinuously photographed to connect recognized results together to one,a function of “connecting” is selected in the function menu displayscreen 76. When the user selects a column of “connecting” to focus itand presses down the “select” button assigned to the function key PF1,the preview display screen 72 is displayed again so that the characterstring can be photographed. Then, when the photographing and characterrecognizing processes of the character string as the object to berecognized are completed subsequently to the previous time, the previewdisplay screen is switched to the recognized character display screen75. Then, the character string recognized at this time is additionallydisplayed. In recognizing the character, the character may be recognizedfor a plurality of images or after the plurality of images are connectedtogether to one image, the character may be recognized.

In the function menu display screen 76, when the user selects eachfunction except “connecting” and presses down the “select” buttonassigned to the function key PF1, a function screen 77 corresponding toeach function is displayed so that each function can be performed. When“editing” is selected, a text editor is activated to display an editingscreen. The user can correct an erroneously recognized character in theediting screen. When “copying” is selected, a copying function is calledto display a copying range selecting screen. Thus, the user can copy thecharacter data a range of which is designated and use the copiedcharacter data in other character input screen. When “storing” isselected, a storing process of the character data of the recognizedcharacter string is carried out. When the storing process is completed,a storing operation completion informing screen such as “stored” isdisplayed. Thus, the recognized character string obtained by thecharacter recognizing process at this time is stored in the column ofthe “non-registered” selected in the main menu screen 71.

Further, when “newly registering in a telephone directory” or“additionally registering in a telephone directory” is selected, atelephone directory registering screen is displayed so that the user canregister the recognized character strings such as the telephone number,the mail address, a name, or the like in desired columns of thetelephone directory. When “registering a book mark” is selected, a bookmark registering screen is displayed so that the user can register therecognized character string such as the URL as a book mark.

In the recognized character display screen 75, when the user selects thecharacter string of any of the telephone number, the mail address andthe URL, and presses down the “select” button assigned to the functionkey PF1, a using screen 78 corresponding to the type of each of thecharacter strings is displayed. When the telephone number is selected,as shown in FIG. 13(C), an audio transmitting screen 78 a is displayedas the using screen. In the audio transmitting screen 78 a, an audiotransmission to the telephone number whose character is recognized canbe carried out by a simple operation. When the URL is selected, as shownin FIG. 13(D), an internet connecting screen 78 b is displayed as theusing screen. Thus, an internet connection by the URL whose character isrecognized can be carried out by a simple operation. When the mailaddress is selected, as shown in FIG. 13(E), a mail preparing screen 78c is displayed as the using screen. Thus, the preparation of anelectronic mail to the mail address whose character is recognized can becarried out by a simple operation.

As described above, in the third embodiment as in the first and secondembodiments, the character frame for recognizing the character isdisplayed on the preview display screen upon photographing the object tobe recognized to photograph the character image of the object to berecognized. Thus, the character of the image in the frame is recognizedso that a load applied to the character recognizing process can bereduced.

Further, the recognizing mode is selected to perform the characterrecognizing process in accordance with the type of a preset character asthe object to be recognized, so that a character recognizing accuracycan be improved. Further, on the recognized character display screen,the character data of a plurality of types are arranged and displayed asthe recognized results. Thus, the character data can be divided forprescribed character units and individually selected, used and stored.Thus, the recognized character data corresponding to the types of thecharacter strings can be easily employed, for instance, the audiotransmission in the case of the telephone number, the preparation of themail in the case of the mail address or the like.

The present invention is not limited to the above-described embodimentsand may be put into practice in various ways within a scope withoutdeparting its gist.

The present invention is described in detail by referring to thespecific embodiments. However, it is to be understood to a person withordinary skill in the art that various changes or modifications may beapplied to the invention without departing the spirit and the scope ofthe present invention.

This application is based on Japanese Patent Application No. 2002-230312filed on Aug. 7, 2002 and Japanese Patent Application No. 2003-133171filed on May 12, 2003 and the contents of them are incorporated hereinas references.

INDUSTRIAL APPLICABILITY

As described above, according to the present invention, a characterrecognition processing device and a character recognition processingmethod can be provided which can improve a rate of recognition and arecognition speed, can reduce the degree of a specification required fora device forming a photographing device or a system and can decreaseconsumed electric power and a cost.

1. A character recognition processing device, comprising: aphotographing unit which photographs a plurality of character imagesusing a continuous photographing operation in which a continuous stillimage is captured and automatically divided into the plurality ofcharacter images, wherein each character image comprises a plurality ofcharacters; an image fetching unit, which fetches image data of theplurality of character images as objects to be recognized, the pluralityof character images being photographed for recognizing individualcharacters of the plurality of characters in each character image; acursor information output unit, which outputs cursor positioninformation showing a position of a character frame, wherein thecharacter frame includes vertical marks and horizontal lines to be usedfor separating each of the individual characters of the plurality ofcharacters in each character image from each other; a display thatsimultaneously displays a cursor, which includes the character frame,with the continuous still image at the time of capturing the continuousstill image; a layout analyzing unit, which collates the cursor positioninformation with the fetched image data of the plurality of characterimages to analyze an arrangement of said individual characters; acharacter cutting unit, which extracts said individual characters on thebasis of the analyzed result of the layout analyzing unit; and acharacter recognizing unit, which recognizes the extracted individualcharacters and converts the extracted individual characters to characterinformation, wherein said character frame including vertical marks andhorizontal lines is displayed with the continuous still image prior tosaid character recognizing unit recognizing the individual characters.2. The character recognition processing device according to claim 1,wherein the image fetching unit fetches the image data of the characterimages respectively by a prescribed area from all of the characterimages; and wherein the layout analyzing unit collates the cursorposition information with the fetched image data of each of theplurality of character images separately.
 3. The character recognitionprocessing device according to claim 1, wherein the image fetching unitfetches the image data of the character images respectively by aprescribed area from all of the character images; and wherein the layoutanalyzing unit collates the cursor position information with the imagedata in which the plurality of fetched character images are connectedtogether.
 4. A portable terminal device having the character recognitionprocessing device according to any one of claims 1 to
 3. 5. A portableterminal device, comprising: a photographing unit, which photographs aplurality of character images using a continuous photographing operationin which a continuous still image is captured and automatically dividedinto the plurality of character images, wherein each one of theplurality of character images comprises a plurality of characters; animage fetching unit, which fetches image data of the photographedplurality of character images; a cursor information output unit, whichoutputs cursor position information showing a position of a characterframe, wherein the character frame includes vertical marks andhorizontal lines to be used for separating each of the individualcharacters of the plurality of characters in each character image; adisplay that simultaneously displays a cursor, which includes thecharacter frame, with the continuous still image at the time ofcapturing the continuous still image; a layout analyzing unit, whichcollates the cursor position information with the fetched image data ofthe plurality of character images to analyze the arrangement of saidindividual characters; a character extracting unit, which extracts saidindividual characters on the basis of the analyzed result of the layoutanalyzing unit; and a character recognizing unit, which recognizes theextracted individual characters and converts the extracted individualcharacters to character information, wherein said character frameincluding vertical marks and horizontal lines is displayed with thecontinuous still image prior to said character recognizing unitrecognizing the individual characters.
 6. The portable terminal deviceaccording to claim 5, further comprising a recognized character displayunit which displays, on said display, the character information as arecognized result by the character recognizing unit.
 7. The portableterminal device according to claim 6, wherein the recognized characterdisplay unit individually selectively displays the character informationas the recognized result by a prescribed character unit.
 8. The portableterminal device according to claim 5, further comprising a recognizedcharacter storing unit which stores the character information as therecognized result obtained by the character recognizing unit.
 9. Theportable terminal device according to claim 8, wherein the recognizedcharacter storing unit stores the character information in a recognizedcharacter storing area.
 10. The portable terminal device according toclaim 8, wherein the recognized character storing unit registers thecharacter information in a data base corresponding to a type of eachcharacter information when the type of the character information is anyone of a telephone number, a mail address and a URL (Uniform ResourceLocation).
 11. The portable terminal device according to claim 5,further comprising a recognized character utilizing unit which utilizesthe character information as the recognized result obtained by thecharacter recognizing unit in accordance with a type of the characterinformation.
 12. The portable terminal device according to claim 11,wherein when the type of the character information is a telephonenumber, the recognized character utilizing unit displays a transmittingscreen to the telephone number.
 13. The portable telephone terminaldevice according to claim 11, wherein when the type of the characterinformation is a mail address, the recognized character utilizing unitdisplays a preparing screen for an electronic mail to the mail address.14. The portable terminal device according to claim 11, wherein when thetype of the character information is a URL (Uniform Resource Locator),the recognized character utilizing unit displays a network connectingscreen to the URL.
 15. The portable terminal device according to claim5, wherein before the image fetching unit fetches the image data of theplurality of character images, the image data of the plurality ofcharacter images are connected together; and wherein the characterrecognizing unit recognizes the extracted individual characters of theimage data of the connected character images to convert the image dataof the connected character images to the character information.
 16. Theportable terminal device according to claim 5, wherein the characterrecognizing unit separately recognizes each of the extracted individualcharacters.
 17. The portable terminal device according to claim 5,wherein the character recognizing unit has a plurality of recognizingmodes that each correspond with a type of character information; andwherein the character recognizing unit carries out a characterrecognizing process suitable for the corresponding type of characterinformation in accordance with a preset recognizing mode.
 18. Acharacter recognition processing method performed by a portable terminaldevice, comprising the steps of: photographing, by the portable terminaldevice, a plurality of character images, each of which comprises aplurality of characters, while simultaneously displaying each respectivecharacter image together with a cursor that includes a character framefor recognizing a character, wherein the plurality of character imagesare photographed using a continuous photographing operation in which acontinuous still image is captured and automatically divided into theplurality of character images; fetching image data of the photographedplurality of character images; outputting cursor position informationshowing the position of the character frame, wherein the character frameincludes vertical marks and horizontal lines to be used for separatingeach of the individual characters of the plurality of characters in eachcharacter image; collating the cursor position information with thefetched image data of the plurality of character images to analyze thearrangement of said individual characters; extracting said individualcharacters on the basis of the analyzed result of the arrangement ofsaid individual characters; and recognizing the extracted individualcharacters images as the characters and converting the extractedindividual characters to character information, wherein said outputtingthe character frame including vertical marks and horizontal lines withthe continuous still image is performed prior to recognizing theextracted individual character images as characters.
 19. A characterrecognition processing program in which the respective steps defined inclaim 18 are executed by a computer running a program, wherein saidprogram is stored within a non-transitory computer readable medium thatis accessible by the computer.